Systems and methods for ultrasonic, millimeter wave and hybrid sensing

ABSTRACT

A close-range motion detector has at least one transmitter, at least one receiver, and at least one more transmitter or receiver. The transmitter(s) transmit, and the receiver(s) receive signals in one of the ultrasonic or mm-wave ranges. Multiple transmitters or receivers are spaced apart from one-another along a plane, and transmission of a signal takes place at a known time. Echos of the signal that bounce of a scatterer are received and digitized during a receive window, and the time-of-flight is determined using CAF. Time scaling may be determined as well, and may be determined using CAF. The determined time-of-flight is used to determine an X-Y-coordinate for the scatterer, and its motion (e.g., velocity) can be determined, which are output. In an embodiment, a such a close-range motion detector can be implemented on the side of a smart-watch, making a virtual writing surface on the back of a user&#39;s hand.

This application is a non-provisional of and claims priority to U.S.Provisional Patent Application No. 62/379,627 file on Aug. 25, 2016 andentitled SYSTEMS AND METHODS FOR ULTRASONIC, MILLIMETER WAVE AND HYBRIDSENSING.

This application includes material which is subject to copyrightprotection. The copyright owner has no objection to the facsimilereproduction by anyone of the patent disclosure, as it appears in thePatent and Trademark Office files or records, but otherwise reserves allcopyright rights whatsoever.

FIELD

The disclosed systems and methods relate in general to the field ofultrasonic and millimeter wave sensing, and in particular to ashort-range (e.g., 10 cm) sensor array to detect touch in a detectionarea.

SUMMARY OF THE INVENTIONS

It is an objective to detect various kinds of gestures that will dependon the application in question. Some example gestures might be a fingerdrawing on the back of a hand, or a hand gesturing above a mobile phone.The gestures could be in one, two or three dimensions (not including the“time” dimension). Detected objects may be in motion or they may bestationary, and detection methods may be able to distinguish stationarytargets from moving ones, or may only be able to distinguish movingtargets.

BRIEF DESCRIPTION OF THE DRAWINGS

Objects, features, and advantages of the invention will be apparent fromthe following more particular description of preferred embodiments asillustrated in the accompanying drawings, in which reference charactersrefer to the same parts throughout the various views. The drawings arenot necessarily to scale, emphasis instead being placed uponillustrating principles of the invention. Although example embodimentsand associated data are disclosed for the purpose of illustrating theinvention, other embodiments and associated data will be apparent to aperson of skill in the art, in view of this disclosure, withoutdeparting from the scope and spirit of the disclosure herein.

FIG. 1 provides an illustration of an embodiment of signal receptiongeometry of ultrasonic or millimeter wave (mm-wave) sensors for 2-Dposition and relative velocity estimation.

FIG. 2 is an illustration of a trajectory of the scatterer (target) forcross motion on a 2-D surface in connection with one embodiment of theinvention.

FIGS. 3A-C are illustrations of an exemplary transmitted signal andcorresponding signals received at two different receivers in connectionwith an embodiment of the invention.

FIGS. 4A-C illustrates a transmitted signal segment with a window lengthof 1.5 msec, a received signal segment of 3 msec and an exemplary bestmatch.

FIGS. 5A-J illustrates measurements of CAF for BW at differing bandwidthvalues.

FIGS. 6A-J illustrates estimated and actual 2-D positions for BW atdiffering bandwidth values for course hypothetical relative velocitygrid in accordance with one embodiment.

FIGS. 7A-J illustrates estimated and actual 2-D positions for BW atdiffering bandwidth values for fine hypothetical relative velocity gridin accordance with another embodiment.

FIGS. 8A-D shows an embodiment of the RMSE in relative range estimationresults for varying window times and bandwidths for combinations of fineand course hypothetical relative range and velocity grids.

FIGS. 9A-D shows another embodiment of the RMSE in relative velocityestimation results for varying window times and bandwidths forcombinations of fine and course hypothetical relative range and velocitygrids.

FIG. 10 illustrates simulation time per measurement for varying windowtimes.

FIG. 11 is a graph showing a relationship between the number of windows(measurements) and window times across hypothetical relative range andvelocity grid combinations.

FIG. 12 is a graph showing a relationship hypothetical relative velocitystep size for varying window times for combinations of fine and coarsehypothetical relative range and velocity grids.

FIG. 13 illustrates transmitters and receivers in a wrist worn device inaccordance with one embodiment of the invention.

FIGS. 14A-B illustrate a use case for a wrist worn device in accordancewith one embodiment of the invention.

FIG. 15 provides a sample list of in-air gestures that can be deployedin connection with an exemplary embodiment of the invention.

FIG. 16 illustrates another use case in accordance with anotherembodiment of the invention.

FIG. 17 is a schematic illustration of a touch surface in accordancewith an embodiment of the invention.

DETAILED DESCRIPTION

The present application contemplates various embodiments of ultrasonicor millimeter-wave technology designed for short-range, human-computerinteraction (HCI) applications that are implemented on mobile devices orpartially mobile devices. In an embodiment, it is desirable that thetechnology have the following qualities: high spatial resolution (in anembodiment, on the order of 1 mm of better); low latency (in anembodiment, less than 100 milliseconds, in an embodiment, less than 10milliseconds, or better); low power (so as not to tax batteries; andsmall size (so that it can fit in small, portable devices, e.g.,watches, mobile phones, wearables, etc.).

In an embodiment, wavelengths are close or nearly the same forultrasonic frequencies of interest and for millimeter wave RFfrequencies. In an embodiment, ultrasonic frequencies may be used. In anembodiment, millimeter wave RF may be used. In an embodiment, acombination of millimeter wave RF and ultrasonic signals with similarwaveforms may be used. Because light travels in the neighborhood of onemillion times faster than sound, RF frequencies that are about a milliontimes the sound frequencies will have the same wavelength and relateddetection characteristics, thus, in an embodiment, similaremitter/receiver geometry may be used for millimeter wave RF andultrasonic frequencies. For example, a 60 kHz acoustic waveform and a 60GHz RF waveform both have a wavelength of about five millimeters. Thus,embodiments disclosed herein that use ultrasonic waves can beimplemented using RF waves and vice versa.

Emitter and Receiver Geometry

The geometry of the emitters and receivers will vary according to theparticular application and the spatial and temporal precision requiredby a particular application. Geometric dilution of precision applies, soin an embodiment, the emitters and receivers will need to be placed inpositions that provide an acceptable dilution of precision. In anembodiment, emitters and receivers are placed in monostatic pairs. In anembodiment, emitters and receivers use a distinct frequency, code ortime slot per pair. In an embodiment, emitters and receivers areindependent (fully multistatic). In an embodiment, a combination of theforegoing emitter and receiver deployments are used.

Signal Structure

In an embodiment, a signal comprised of pulses is used. A signalcomprised of pulses (narrow features) in the time domain tends to haveexcellent range resolution, but poor velocity (i.e. Doppler) resolution.In an embodiment, a signal comprised of signal comprised of narrowfeatures in the frequency domain is used. A signal comprised of narrowfeatures in the frequency domain (e.g. sinusoids) tends to haveexcellent velocity resolution, but poor range resolution. In anembodiment, a noise-like signal is used. A noise-like signal can haverange resolution and velocity resolution that are both excellent.

It has been discovered that range precision (i.e., minimum measurabledifference in target range) of a signal deployed in an embodiment of thepresent invention is proportional to the propagation velocity of thesignal and inversely proportional to the signal's bandwidth (BW) and tothe square root of its signal-to-noise ratio (SNR). The followingformula describes that relationship.

$\begin{matrix}{r_{prec} \approx \frac{v_{p}}{B\; W\sqrt{S\; N\; R}}} & (1)\end{matrix}$

It will be apparent to one of skill in the art that the higher thebandwidth and SNR, and the lower the propagation velocity, the moreprecisely range is measured.

It has also been discovered that the velocity precision (i.e minimummeasurable difference in target velocity) of a signal deployed in anembodiment of the present invention is proportional to the propagationvelocity of the signal and inversely proportional to the signal'sduration and to the square root of its signal-to-noise ratio. Thefollowing formula describes that relationship.

$\begin{matrix}{v_{prec} \approx \frac{v_{p}}{T\sqrt{S\; N\; R}}} & (2)\end{matrix}$

In both cases above, the signal-to-noise ratio (SNR) corresponds topower: it is the ratio of the signal power to the noise power.

In an embodiment, precision in both measured range and measured rangerate are increased to the extent possible for a given implementation. Asused above, precision means having a smaller measurable difference; andmeasured range rate refers to velocity in the range direction. In anembodiment, precision is achieved by decreasing decrease r_(pre) andv_(prec). In an embodiment, precision can be achieved by decreasingv_(p), however the propagation velocity is almost always fixed for aparticular technology (v_(p)=c for RF and v_(p) is the speed of soundfor ultrasonics and acoustics). In an embodiment, increasing the SNRwill help in both cases, but the benefit only increases with the squareroot of the amount of power put in and, in many applications, power islimited. In an embodiment, a waveform that is both broadband and has along time duration allows simultaneous measurement of both the range andrange rate (velocity) with good precision. In an embodiment, anoise-like waveform with sufficient bandwidth, transmitted (andreceived) over a sufficient time period will yield the requiredprecision.

In an embodiment, to achieve a spatial precision on the order of onemillimeter without going to a high SNR, the bandwidth is on the order of300 GHz for radio waves traveling at c, and the bandwidth is on theorder of 300 kHz for acoustic waves traveling at the speed of sound. Inan embodiment, the center frequency is at least half of the bandwidth.The specific numbers presented for this embodiment are estimates, andare affected by other factors, such as the SNR and the geometry of thetransmitter and receiver. In an embodiment, a monostatic geometry isemployed, with waves traveling out to a target and the reflectiontraveling back the same way, the range is essentially doubled so thatonly half the bandwidth is required.

In some embodiments (including where received noise is independent fromthe signal, additive, white and Gaussian) a matched filter may beemployed. In an embodiment, a matched filter may be optimal or may bethe most practical method for recovering the received signal.

Due to the velocity of propagation of the signal and the distance to thetarget the delay between transmission of a signal and its receptionshould be measured. In an embodiment, the delay between transmission ofa signal and its reception is measured accurately. In an embodiment, thedelay between transmission of a signal and its reception is measured asaccurately as possible. The Doppler shift of the signal is a metric tomeasure the relative velocity of the target, i.e. the velocity of thescatterer in the direction of the phase center of the emitter andreceiver. Thus, in an embodiment, the Doppler shift of the signal ismeasured accurately. In an embodiment, the Doppler shift of the signalis measured as accurately as possible. In an embodiment, both the delayand Doppler shift are measured by calculating the cross-ambiguityfunction (CAF). In an embodiment, the CAF produces a two-dimensionaloutput for each transmitter/receiver pair, the output showing theresponse of the matched filter for each possible Doppler shift and eachpossible time shift (delay) that the signal experiences in travelingfrom the transmitter to the receiver, via any targets. The Doppler shiftand time shift correspond to the relative velocities and positions ofany targets in the field of view of the transmitter and receiver, andprovides information about the relative position and velocity of thetransmitter to the receiver (if these are in each other's fields ofview).

Synched Data I/O

In an embodiment, a device employs Synched Data I/O (hereinafter TSDIO)which will, in a synchronized manner, send data from a host computer viaUSB to a set of digital-to-analog converters (DACs). As the DACs outputthe analog values corresponding to their digital inputs, a set ofanalog-to-digital converters (ADCs) samples its analog inputssimultaneously, so that the analog samples (both in and out) all occurat the same time and at the same rate. TSDIO will performrapid-iteration experiments with bidirectional, multistatic ultrasonicsonar systems. In an embodiment, TSDIO will need to be flexible and haveadjustable parameters.

In an embodiment, TSDIO may be implemented as a single mixed-signalASIC. In an embodiment, TSDIO may be implemented as a single circuitboard.

In an embodiment, TSDIO may be implemented as two separate circuitboards: a digital board comprised of the USB communication, FPGA gluelogic, power supplies and timing circuitry and the DACs and ADCs, and ananalog board to perform amplification, level-shifting and general signalconditioning, and is connected to the analog inputs and outputs of theADCs and DACs. Separating the digital and analog portions (i) enablesdifferent analog boards to be easily and quickly designed andfabricated, and (ii) allows the digital portion to be re-used in a widevariety of applications. In an embodiment, the two boards are connectedtogether with an electrically reliable board-to-board connector to allowreliable electrical operation. In an embodiment, the two boards areconnected together with a mechanically reliable connection to alleviatemechanical stress on the board-to-board connector. In an embodiment, thetwo boards are connected together with an electrically and mechanicallyreliable connector to allow both reliable electrical operation and toalleviate mechanical stress on the board-to-board connector.

USB Communication

In an embodiment, the digital board communicates with a host computervia a wired or wireless communications channel. In an embodiment, thedigital board communicates with a host computer employing a standardcommunication channel such as, USB. In an embodiment, the digital boardcommunicates with a host computer using USB 3.0, e.g., via a Cypress FX3microcontroller. For wired communications, in an embodiment, the digitalboard provides some method for strain relief of the cable, e.g., USBcable. In an embodiment, USB bulk or isochronous data transfer will beused. In an embodiment, a facility for appropriate bidirectional controland reporting of status and setup parameters using USB bulk or interrupttransfers is provided. In an embodiment, either or both of the digitaland the analog boards can provide a self-identification feature so thata host computer can discover what the board type of connected boards,e.g., model, version, as well as an optional unique serial number foreach board instance.

FPGA

In an embodiment, the digital board can use an FPGA as “glue” logic. Inan embodiment, FPGA programming can be in Verilog.

DACS

In an embodiment, the TSDIO has at least four digital-to-analogconverters (DACs). In an embodiment, the TSDIO has at least eight DACs.In an embodiment, the TSDIO DACs are at least 16-bit, or at least20-bit. In an embodiment, the TSDIO DACs are 24-bit DACs. In anembodiment, the TSDIO DACs are at least 32-bit. In an embodiment, theDACs convert digital data at the rate necessary to perform as describedherein. In an embodiment, the DACs convert digital data at a rate of toat least 5 Msps. In an embodiment, the DACs output samplessimultaneously. In an embodiment, the DACs' sample rates are dynamicallychangeable under program control via USB. In an embodiment, thegranularity of the sample rate setting is e.g., 10 kHz, however,generally, a finer sample rate may provide better results other thingsbeing equal. In an embodiment, it is possible to specify, under programcontrol (e.g., via USB), that data is to be sent to only a subset of theDACs and that the others should output a default value or another presetvalue. In an embodiment data is sent to a subset of the DACs (e.g., andhave the others output a default or preset value) to trade off DAC speedagainst the number of simultaneous analog outputs. In an embodiment,data is sent to a subset of the DACs to conserve USB bandwidth.

ADCs

In an embodiment, the TSDIO has at least four, or at least eight, 16-bitor better analog-to-digital converters (ADCs). In an embodiment, ADCssample their analog inputs at a rate of to at least 5 Msps. In anembodiment, all ADCs sample their inputs simultaneously, and the samplerate may be changed dynamically under program control (e.g., via USB).The granularity of the sample rate setting should be reasonable. In anembodiment, the granularity of the sample rate is no larger than 10 kHz.All other things being equal, finer granularity is better. In anembodiment, data is received from only a subset of the ADCs. In anembodiment, the ADCs from which data is received is specified underprogram control (e.g., via USB). In an embodiment data is received froma subset of the ADCs to trade off ADC speed against the number ofsimultaneous analog inputs. In an embodiment, data is sent to a subsetof the ADCs to conserve USB bandwidth.

Clocking and Timing

In an embodiment, the TSDIO uses a low-jitter (i.e. low phase noise)clock source. In an embodiment, the TSDIO has provisions for the use ofan external clock source. In an embodiment an external, RF-qualityphase-locked loop is employed to generate frequencies from the clocksource. The PLLs internal to FPGAs are generally not suitable for thisapplication. In an embodiment, multiple TSDIOs are synchronized. In anembodiment, multiple TSDIOs are synchronized by sharing a clock signal.In an embodiment, multiple TSDIOs are synchronized by having a commonexternal clock signal. In an embodiment, multiple TSDIOs aresynchronized by having one TSDIO share an internal clock signal with oneor more other TDSIOs.

Power Supply

In an embodiment, the TSDIO is powered entirely through the USB port. Inan embodiment, alternate power ports are provided for the analog boardand analog portions of DACs and ADCs. In an embodiment, alternate powerports are provided for the analog board and analog portions of DACs andADCs so that they can be powered from bench supplies for testingpurposes. In an embodiment, jumpers can be used for switching betweenpower from the USB port and external power. In an embodiment, jumperscan be used for temporary connections. In an embodiment, placement andremoval of zero-ohm resistors can be used for switching between powerfrom the USB port and external power. In an embodiment, the zero-ohmresistors are physically large enough that they can be placed or removedmanually, e.g., by using hand or by hand using soldering tools. In anembodiment, zero-ohm resistors are not smaller than 0805 SMT parts.

Noise Issues

In an embodiment, the TSDIO employs low noise design and layouttechniques. To this end, the design should provide good isolationbetween the DAC and ADC portions and their downstream connections on theanalog board. For all analog supply lines, any power sourced from aDC-DC or other noisy power supply should be followed by a LDO linearregulator. In an embodiment, a low-noise LDO linear regulator isemployed.

Synchronization

In an embodiment, the TSDIO board operates in a synchronized manner.When the TSDIO board operates in a synchronized manner it is arelatively simple task to calculate and verify latencies in the system.E.g., ADC sampling is simultaneous with DAC output. In an embodiment, tomitigate noise, DC-DC converters are run at frequencies synchronized tothe main system clock.

In an embodiment, data sent to DACs (e.g., through USB) is associatedwith a timestamp or arbitrary parameter (i.e., a changing/evolvingparameter), and data sampled at the ADCs is associated with a timestampor arbitrary parameter. In an embodiment, the time stamp or parametermay be used as a tag for the ADC data that is sampled at the same momentthat the time-stamped DAC data is converted to analog. In an embodiment,associating the ADC and/or DAC with a timestamp or parameter data allowsdetermination (e.g., by software on the host computer) of which sampleson the input and output sides occurred at the same time. In anembodiment, associating the ADC and/or DAC data with a timestamp orparameter permits accurate determination of relative timing of sampledata.

Analog Board

In a multiple-board embodiment, the analog board may be attached to thedigital board via a board-to-board connector, and DAC outputs are routedfrom the digital board to the analog board. ADC inputs are routed fromthe analog board to the digital board. It will be apparent to one ofskill in the art in view of this disclosure that power supplies are alsorouted to the analog and/or digital boards as necessary. In anembodiment, either the DAC or ADC analog values (or both) are transmitin differential form one board to the other (e.g., over theboard-to-board connector). In an embodiment, voltage references for DACsand ADCs are routed to the analog board. In an embodiment, a digitalport (e.g., SPI) is routed between the analog and digital board to allowanalog board parameters to be set under program control (e.g., via theUSB port).

In an embodiment, the analog board supports the same number ofsingle-ended analog inputs as there are ADC inputs on the digital board.In an embodiment, all have a reasonably high impedance and support atleast a 2 MHz analog. In an embodiment, the analog board should supportthe same number of single-ended analog outputs as there are DAC outputson the digital board. In an embodiment, all have a reasonably low outputimpedance and support at least a 2 MHz analog bandwidth. In anembodiment, the input voltage range of the analog inputs is mapped ontothe full range of the ADCs. Similarly, in an embodiment, the outputvoltage range of the analog outputs should be mapped onto the full rangeof the DACs. Thus, in an embodiment:

-   -   Input voltage range settable under program control from 1 volt        peak-to-peak to 5 volts peak-to-peak; input impedance no lower        than 1 megaohm.    -   Output voltage range settable under program control from 1 volt        peak-to-peak to 12 volts peak to peak; output impedance no        higher than 100 ohms.        Construction

In an embodiment, the TSDIO device uses surface mount construction wherepractical. In an embodiment, a case for the device is optional. In anembodiment, the board set is configured to use rubber feet. In anembodiment, mounting holes for a case may be provided in the board set.

Mechanical Layout

In an embodiment, the TSDIO device is as small as practicable.Generally, the size of the board set is not the highest priority. In anembodiment, the digital and analog boards have facilities for propermechanical attachment to each other so as not to stress theboard-to-board connector.

Test Features

In an embodiment, the TSDIO device employs appropriate test anddebugging features including a JTAG chain on the chips that support it.In an embodiment, the boards include test points for ground and thevarious power supplies and signals that would be useful for debugging.

Driver and Demo Software

In an embodiment, the TSDIO device may be configured to be used onWindows or Linux platforms, or both.

2-D Position and Relative Velocity Estimation Using Ultrasonic Sensorsfor Close Range Scatterer Motion Recognition

Ultrasonic transducers can use sound signals for sensing applications.Typical selected frequencies for ultrasonic transducers are above thehearing limit of humans, from 20 KHz and up to several gigahertz. Thehigher the frequency the shorter the wavelength; and in sensingapplications, the higher the frequency the greater the ability to detectsmaller objects. In an embodiment, higher frequency waveforms can beused effectively for short distance sensing without the need of a largetransmission power.

Relative range and velocity of a scatterer can be predicted using thetransmitted and received waveforms. A scatterer can be any object thatreflects a measurable quantity of the transmitted waveforms, in thedirection of reception or in a direction that is further reflected inthe direction of reception. A transmitting transducer transmits signalsthat travel at speed of sound. The transmitted signals bounce offscatterers in their range. A receiving transducer collects reflectedsignals (echoes). By synchronizing the received signals with thetransmitted signal, the time delay between the transmission andreception can be determined. The time delay can be used to estimate thescatterer's range relative to the transducers. The received signalstretches or compresses (scales) in time domain with respect to thevelocity of the scatterer, which is known as the Doppler effect. In anembodiment, the relative velocity that caused a time scaling effect canbe measured by processing the received signal.

More than one receiver is required to determine a coordinate rather thana relative position of a scatterer. In an embodiment, the 2-D positionof a scatterer on a flat surface within a short range is determinedusing at least two receivers and one transmitter.

Turning to FIG. 1, an embodiment of a signal reception geometry isshown. wherein one receiver Rx1 and transmitter Tx1 are collocated andanother receiver Rx2 is spaced apart therefrom at a fixed distance alonga hypothetical y-axis. Reflections of the transmitted signal may bereceived by the receivers Rx1 and Rx2, which form a monostatic andbistatic configuration with the transmitter Tx1, respectively. Thisgeometry permits estimation of the 2-D (x-coordinate, y-coordinate)position of the scatter and its radial velocity with respect to thephase center of the transducers. As shown in FIG. 1, the Tx1-Rx1 pairforms a monostatic sensing geometry, whereas Tx1-Rx2 forms a bistaticsensing geometry. Rx2 is separated from Tx1-Rx1 with a distance of Δy.In an embodiment, transducer Tx1-Rx1 and receiver Rx2 are located onwearable such as a smart watch, and oriented to permit the wrist or backof the hand to operating as a virtual writing surface. TransducerTx1-Rx1 and receiver Rx2 require separation in the y direction toprovide data that can be resolved to a two-dimensional coordinate on thex-y plane. In an embodiment, transducer Tx1-Rx1 and receiver Rx2 areseparated by a Δy of greater than 1 cm. In an embodiment, transducerTx1-Rx1 and receiver Rx2 are separated by a Δy of greater than 1.5 cm.In an embodiment, transducer Tx1-Rx1 and receiver Rx2 are separated by aΔy of 1.706 cm. In an embodiment, transducer Tx1-Rx1 and receiver Rx2are separated by a Δv of not more than 2 cm. In an embodiment,transducer Tx1-Rx1 and receiver Rx2 are separated by a Δy of over 2 cm.Higher separation between receivers decreases the margin of error ingeometrical calculation to estimate 2-D positions. Thus, in anembodiment, even for a small device such as a watch, a Δy of over 2 cmmay be used. For larger devices (e.g., tablets), a Δy of oversubstantially more than 2 cm may be used.

For the purposes of discussion herein, a Δy of 1.706 cm is from time totime used in the calculations behind estimation e.g., of the relativeposition and velocity by using the transmitting and received ultrasonicsignals. This is for exemplary purposes only, and is not intended tolimit the scope of this disclosure, which is intended to be limited onlyby the appended claims. Simulation results are presented to demonstrateaccuracy in 2-D position, relative position and velocity estimation fordifferent sampling frequencies, bandwidths, window time, hypotheticalvelocity resolution, scatterer position and velocities. Again, thesimulation results are illustrative examples only, and are not intendedto limit the scope of this disclosure, which is intended to be limitedonly by the appended claims.

Received Signal Equation

In an illustrative embodiment, the relative range between a movingscatterer and the transducers can be described as follows:R(t)=R ₀ +vt and R(0)=R ₀,  (1)where v is the velocity of the scatterer along the look-direction of thetransducer, t is the time, and R₀ is the initial range of the scatterert=0.

For a monostatic transducer, i.e., where the transmitter and receiverare collocated, the round trip range (RTR) can be defined for a waveformasRTR=cT(t)=2R[t−T(t)/2],  (2)where T(t) is the round trip time (RTT) for the waveform received attime t and which is reflected from the scatterer at time t−T(t)/2. RTTcan be solved by plugging in the R(t) in (1) into (2):

$\begin{matrix}{{R\; T\; T} = {{T(t)} = {{{2{R_{0}/\left( {c + v} \right)}} + {2{{vt}/\left( {c + v} \right)}}} = {{{\left( {2{R_{0}/c}} \right)/\left( {1 + {v/c}} \right)} + {2{{vt}/\left( {c + v} \right)}}} \cong {{2{R_{0}/c}} + {2{{vt}/\left( {c + v} \right)}}}}}}} & (3)\end{matrix}$where c is the speed of sound (approx. ^(˜)340 m/sec). In an embodiment,the approximation that 2R₀/(c+v)≈2R₀/c is sufficient for typicalscatterer (e.g., finger on the back of a hand) velocities. This alsomeans that R₀ points out to the initial range of the scatterer where thewaves hit it. Since v is constant and does not depend on time, oneassumes that the scatterer does not change during the reflection period.In an embodiment, the reflection period is equal to the time window thattransmitter signals are divided to and typically very small, which makesthe constant velocity assumption reasonable.

Defining the transmitted signal as s(t), the received signal S_(R)(t)can be represented in terms of s(t) as follows:S _(R)(t)=αs(t−T(t))=αs(t(c−v)/(c+v)−2R ₀ /c),  (4)where α is a complex constant that accounts for the reflectionattenuation. Hence, the received signal S_(R)(t) is the shifted andtime-scaled version of the transmitted signal. The shift is also thetime delay that determines the range of the scatterer at the time ofreflection, and can be denoted with τ=2R₀/c. The time-scaling causes thetransmitted signal to rescale in time according to the Doppler stretchfactor σ=(c+v)/(c−v).Relative Range and Velocity Estimation

As shown in (4), the received signal is the time scaled, delayed, andattenuated version of the transmitted waveform. Neglecting theattenuation and possible environmental noise added to the receivedsignal, given the range and velocity of the scatterer, an exact matchfor the received signal can be reconstructed from the transmittedwaveform. Thus, in an embodiment, by hypothesizing the range andvelocity parameters to synthesize time-delayed and time-scaled versionsof the transmitted signal, the relative range and velocity of ascatterer is predicted. In an embodiment, by correlating thehypothesized signals with the received signal, the highest correlationvalue can be selected as the best match and hence the predicted valuesfor the relative range and velocity to the scatterer. In an embodiment,correlating two signals may be done by multiplying same index values forboth signals and adding them up. A two dimensional grid produced by thehypothesized ranges and velocity produces a matrix of correlation valuesof which the maximum points out to the predicted range and velocity. Werefer to this matrix is as a 2-D cross ambiguity function (CAF) and canbe formulated as follows:

$\begin{matrix}{{{C\; A\;{F\left( {\sigma,\tau} \right)}} = {\sum\limits_{t}{{S_{R}(t)}{s\left( {{t;\sigma},\tau} \right)}}}},{{for}\mspace{14mu} a\mspace{14mu}{number}\mspace{14mu}{of}\mspace{14mu}{hypothesized}\mspace{14mu}\sigma\mspace{14mu}{and}\mspace{14mu}\tau\mspace{14mu}{{values}.}}} & (5)\end{matrix}$

In an embodiment, the maximum value of CAF(σ, τ) provides the estimatesfor σ and τ, and consequently the relative range and velocity.

Monostatic and Bistatic Range Calculations

In the sensing scheme shown in FIG. 1, there are two receivers locatedat different spots, and ranges to be predicted are different for thereceived signal at Rx1 and Rx2. The range of the scatterer to Rx1 andRx2 is defined as R1 and R2, respectively. Tx1 and Rx1 form a monostaticconfiguration because the transmitting and receiving transducers arecollocated. Tx1 and Rx2 form a bistatic configuration because thetransmitter and receiver are separated. We follow the following steps toestimate R1 and R2:

-   -   1. predict the range R1 for the monostatic configuration from        time delay τ=2R₀/c.    -   2. use this range to predict for the range between receiver Rx2        and the scatterer (i.e. R2) by measuring for the time delay        τ=(R1+R2)/c.

The estimated relative velocities are the velocities along the lookdirection of the monostatic and bistatic configuration, whichcorresponds to the direction from midpoint of the transmitting andreceiving transducer to the scatterer of interest.

Determining the 2-D Position of the Scatterer

In an embodiment, after estimating R1 and R2, the triangle formed by R1,R2 and Δy in FIG. 1 can be used to determine the 2-D position of thescatterer. In an embodiment, the 2-D position of the scatterer isdetermined by calculating the interior angles using the cosine rule, andthen using these angles and the predicted ranges to pinpoint to the 2-Dposition of the scatter on the flat surface.

Transmitted Waveform Properties and Factors Affecting the PredictionAccuracy

As an illustrative embodiment (but not intended to limit the generalityof the invention), a continuous-wave noise signal is used. In theillustration, a signal is time stamped and divided into windows (definedby a window time). For each window, the transmitted signal is timedelayed and scaled. The processed waveforms are used to compute the CAFsfor each window time. A description of parameters of the illustrativesignal follows:

-   -   i. Type of signal: The noise signal provides a unique waveform        for every time window in the continuous signal which results in        high peaks in the CAF that provides accurate prediction of the        range and velocity.    -   ii. Sampling Frequency: The sampling frequency of the waveform        determines the resolution in detecting the edge of the        scatterer. Higher sampling frequency provides a finer resolution        in time that waveforms hit the scatterer, which leads to better        resolution in the determined relative range.    -   iii. Window time: The continuous wave signal is divided into        windows defined by a window time. The individual windowed        signals are time delayed and scaled for CAF calculation in order        to estimate the relative position and velocities for each        window. This window time should be chosen carefully. Generally,        it should be long enough to provide a good match for the signal,        but also short enough not to cause degradation in the velocity        estimation.    -   iv. Carrier Frequency and Bandwidth: The carrier frequency        (F_(c)) of the signal determines the wavelength of the waveform        (λ) with the relation λ=c/F_(c). Wavelength should be smaller        than the physical length of the object to be detected. Bandwidth        also has an important role in increasing the strength of peaks        in the CAF. Generally, increasing the bandwidth also increases        the intensity of autocorrelation of the signal, which leads to a        sharper peak when the received signal is matched with the        processed transmitted signal. In an embodiment, this is also        effective in suppression of noise in measurements.        Simulations        Simulation Scenario

In initial illustrative simulations, an attempt was made to stimulate a‘cross’ motion for a finger which is the scatterer of interest. The‘cross’ motion is identified as the scatterer going in horizontal,diagonal and vertical motion, respectively. Turning now to FIG. 2, ascatterer trajectory for cross motion on a 2-D surface is shown. Themotion starts from the left-to-right swipe, followed by the diagonal andvertical swipes. Note that the motion is considered to be on a flatsurface. The axes are given in meters, and motion considered is within10 cm range.

Received Signal Generation

In the illustrative simulations, received signals are artificiallygenerated for the moving scatterer. To do this, the transmitted signalsare time-delayed and time-scaled according to the position of thescatterer after each time window. Hence the transmitted signal isprocessed within each time window and recomposed to achieve the fullreceived signal with respect to the positions of Rx1 and Rx2. FIGS. 3A-Cshow the full transmitted and received signals at Rx1 and Rx2 forF_(s)=176.4 KHz, F_(c)=60 KHz and BW=20 KHz. The total transmission timeis 23.2 msec. Because the scatterer is within a very close range at thestart of the transmission (^(˜)2 cm), the initial delay for receivedsignal is very small. The received signals are delayed and stretched intime differently for Rx1 and Rx2.

2-D CAF Computation

In an embodiment, CAFs are computed according to equation (5) for eachwindow of the signal. The transmitted signal segmented from the timewindow are time-delayed and time-scaled for a set of values of σ and τ.The set of processed windowed transmitted signals are then multipliedelement by element in time domain with the received signal. Each σ and τleads to a value in the CAF of which the maximum points to the predictedrelative range and speed. FIGS. 4A-C show the transmitted signal segmenta window length of 1.5 msec, and received signal segment for 3 msecstarting with the timestamp of the first sample of the transmittedsignal. The predicted relative range and speed for the target is 0.054 mand 8 m/sec. Hence the time-delayed and time-scaled transmitted signalaccording to these values shows the best match with the received signalaccording to their sample positions.

Role of Bandwidth

FIGS. 5A-J show graphical representations of a CAF map for a signal withF_(s)=1.0584 MHz, F_(c)=360 KHz, and bandwidths between 16 KHz and 160KHz. Increasing the bandwidth increases the strength of peaks in CAFmap. Sharpening the peak or peaks may improve prediction accuracy forranges and speeds for scatterer measurements that include noise and/orother interference.

Relative Range and Velocity Estimation Results

In an illustrative embodiment, 3 signals with different sampling (F_(s))and carrier (F_(c)) frequency were used for a set of bandwidth values.The average relative range and velocity error for a set of bandwidths,sampling and carrier frequencies are shown the following Table 1. Theresults are also shown for the window times of 0.9675 msec, 1.5 msec and1.9 msec. For CAF calculations, hypothetical ranges are from 1 cm to 10cm, increasing with 0.1 mm step size. In addition, hypotheticalvelocities between −20 m/sec and 20 m/sec are used, and the results areshown for step sizes of 0.5 m/sec and also 0.1 m/sec.

TABLE 1 The relative positioning and velocity errors averaged in betweenbandwidths for signals with different parameters and target velocitiesFs = 176.4 KHz Fs = 529.2 KHz Fs = 1.0584 MHz Fc = 60 KHz Fc = 180 KHzFc = 360 KHz BW = 2 KHz − 20 KHz BW = 8 KHz − 80 KHz BW = 16 KHz − 160KHz Hypothesized Window time = Window time = Window time = Window time =Window time = Ranges and 1.5 msec (256 1.9 msec (1024 0.9675 msec 1.9msec (2048 0.9675 msec Velocities Errors samples) samples) (512 samples)samples) (1024 samples) Ranges Average 0.0029 m = 0.0107 m = 0.0058 m =0.0337 m = 0.0054 m = 0.01:0.0001:0.1 m Relative 0.29 cm 1.07 cm 0.58 cm3.37 cm (*) 0.54 cm Position Error Velocities Average 0.6646 m/sec0.5186 m/sec 0.4727 m/sec 2.6316 m/sec 0.4654 m/sec −20:0.5:20 Velocity(*) m/sec Error Ranges Average 0.0021 m = 0.00007969 m = 0.000092432 m =0.0024 m = 0.000038984 m = 0.01:0.0001:0.1 m Relative 2.1 mm 0.08 mm0.092 mm 0.24 cm 0.039 mm (**) Position Error Velocities Average 0.6030m/sec 0.4522 m/sec 0.4478 m/sec 0.4668 m/sec 0.4215 m/sec −20:0.1:20Velocity (**) m/sec ErrorThe Results

The results in Table 1 provide some conclusions on prediction accuracyof relative ranges and velocities with respect to the sampling frequency(F_(s)), step size of the hypothesized velocity and window time (samplenumbers). First, it will be apparent to a person of skill in the art inview of this disclosure that prediction accuracy is increased byincreasing the sampling frequency and decreasing the step size of thehypothesized velocity. However, increasing the window time does notnecessarily point to a better estimation performance, as large number ofsamples can cause ambiguity in prediction of the relative velocity whichleads to an erroneous estimated relative position (see e.g., resultsmarked with (*) in Table 1). In accordance with these observations,better results are achieved for F_(s)=1.0584 MHz, F_(c)=360 KHz, windowtime=0.9675 m/sec and with 0.1 m/sec step size for hypotheticalvelocities. These results are marked with (**) in Table 1.

The 2-D position prediction results are also shown for 0.1 m/sec and 0.5m/sec step size for different bandwidths in FIGS. 6A-J and 7A-J,respectively. For 0.5 m/sec step size, the cross shape is formed in itscorrect position as the bandwidth increases. For 0.1 m/sec step size,the cross shape is formed in its correct location for all the bandwidthvalues. FIGS. 6A-J show the estimated and actual position for differentbandwidth values (F_(s)=1.0584 MHz, F_(c)=360 KHz, window time=0.9675m/sec and with 0.5 m/sec step size for hypothetical velocities). Thecross motion shape starts to form in its correct position as thebandwidth increases FIGS. 7A-J show the estimated and actual positionfor different bandwidth values (F_(s)=1.0584 MHz, F_(c)=360 KHz, windowtime=0.9675 m/sec and with 0.1 m/sec step size for hypotheticalvelocities). The cross motion shape is formed in its correct locationfor all the bandwidth values using 0.1 m/sec step size for hypotheticalvelocities.

In the illustrative simulations Matlab's ‘resample’ function was used todo the scaling in time for received signal model and processing oftransmitted signals to calculate the CAFs. The functiondownsamples/upsamples the signal followed by a filter smoothing out theoutput. Accordingly, the result can be regarded as a good approximationfor real world received signal measurements. It should be consideredthat the illustrative simulation results are the result of onerealization of noise, however, results can change for each noise signalgeneration.

Observations

In view of the foregoing, there are several novel contributions to thearea of using ultrasonic signals in order to recognize scatterer motion,including:

-   -   1. Noise, especially having higher bandwidth, is used as a        characteristic signal that leads to sharp peaks in 2-D CAF maps;        and    -   2. 2-D geometry depicted in FIG. 1 is utilized in order to        estimate the 2-D position of the scatterer with a minimal        requirement of one transmitting and two receiving        transducers—the novel geometry and the close range received        signal model are described.

In an embodiment, it is desirable to decrease the computational cost ofthe 2-D CAF calculation. In an embodiment, a two stage (or multi-stage)calculation can be employed, where the stages involve coarse to finestep sizes in hypothetical ranges and velocities. The hypothetical stepsizes are very important since they both affect the computational costand prediction accuracy of the method. In an embodiment, simulations canbe carried to determine the best hypothetical values suiting to aparticular purpose (e.g., gesture recognition by scatterer positioningand motion sensing).

Parameters for Ultrasonic Sensing of Close Range Scatterer Motion

Above, is described an embodiment of an ultrasonic sensing geometry thatprovides acquisition of 2-D position and relative velocity of a singlepoint scatterer moving in close range (<10 cm) to the transducers. Belowis discussed the selection of the transmitted signal and algorithmparameters for our sensing configuration, including specificinvestigation of the role of bandwidth, window time, and hypotheticalrelative range and velocity search grid. In some embodiments, theseparameters have an important role in determining the accuracy andcomputational cost of an embodiment of the disclosed inventive system.

Ultrasonic Sensing Using Pulsed and Continuous Wave

Ultrasonic sensing can be implemented by a pulsed or continuous wave.For pulsed wave, the time between the emission and reception of thepulse bounced off from scatterer (i.e. echo) can be used to predict thedistance to the scatterer. The pulse needs to be received to generatethe next pulse for next distance measurement. This leads to a wait timewhich restricts the speed of refreshing the scatterer location. However,continuous wave (CW) measurements do not require a wait time, rather,the scene is illuminated and echoes are received continuously. CWleverages different aspects of the received signal to measure therelative range and velocity to the scatterer. In an embodiment, thetransmitted CW is time-stamped with different frequency to determine therange to the scatterer by measuring the time lag between the emissionsand echoes. In an embodiment, the attenuation of the received echoes isused to predict the distance to the scatterer. In an embodiment, the CWis utilized to predict the Doppler shift (i.e. relative target velocity)by measuring the frequency shifts between the received and transmittedsignals.

An Embodiment of an Ultrasound Sensing System

In an illustrative ultrasound sensing scheme, CW signals are used. Theapplication of CW signals provides benefit from making continuousmeasurements. In an embodiment, noise signals may be used as thetransmitted waveform. Using noise signals as the transmitted waveformcreates what can be viewed as a natural time stamp on the transmittedsignal because the noise signal is unique for every piece of transmittedsignal defined by a window time. Hence for a time window on thetransmitted signal, the piece of signal can be regarded as a uniquepulse. Similar to pulse wave measurements, the time between the emissionand reception of the windowed transmitted noise is measured to predictthe relative range to the scatterer. In an embodiment, to predict therelative velocity, the Doppler scale parameter in the received signal isalso determined. Hence, in an embodiment, with the aid of noise signals,time-of-flight measurement of the pulsed wave and the Doppler predictionand speed of CW are combined to provide an accurate and fast algorithmfor relative range and velocity estimation.

In an embodiment, the scene is continuously illuminated with thetransmitted noise signal, and echoes bouncing off a scatterer arecontinuously received. The received signal is a time-shifted andtime-scaled version of the transmitted signal, where the time-shift andtime-scale are respectively based on the relative range and velocity ofthe scatterer. Partitioning the transmitted signals into windows, we canpredict the shift and scale for each window time. A set of hypotheticalrelative ranges and velocities can be used to compute time-scaled andtime-shifted versions of the windowed transmitted signal. Correlatingeach of these with the actual received signal, permits prediction of wepredict the relative range and velocity value, e.g., by finding the setwith the highest degree of correlation.

Effects of System Parameters in Sensing Accuracy and Computational Cost

The effect of bandwidth, window time and the hypothetical relative rangeand velocity grid are shown with the simulation results presented inFIGS. 8A-D, 9A-D and 10. FIGS. 8A-D and FIGS. 9A-D show the root meansquared error (RMSE) in relative range and velocity estimationrespectively for varying window times and bandwidths. The window timesare increased from 0.1 msec up to 1.9 msec. The bandwidths are increasedfrom 16 KHz to 160 KHz. The results for the combinations of fine andcoarse hypothetical relative range and velocity grids are shown. FIG. 10shows the simulation time per measurement (window time) for all of thefour combinations. The hypothetical relative range is between 1 cm and10 cm, with the step size of 0.32 mm and 1.6 mm for fine and coarsegrid, respectively. The fine and coarse hypothetical relative velocityis step sized between −20 m/sec and 20 m/sec with the values shown inFIG. 12.

There are several waveform and algorithmic parameters discussed below.

Waveform Parameters

-   -   1) Sampling Frequency: This frequency determines the edge        detection resolution of the scatterer, specifically given by the        formula c/fs, where c is the speed of sound and fs is the        sampling frequency. The resolution gets finer with increased        sampling frequency. For the illustrative embodiment herein, this        frequency is 1.0584 MHz.    -   2) Carrier Frequency: This frequency determines the wavelength        of the signal with the formula c/fc, where c is the speed of        sound and fc is the carrier frequency. Higher carrier frequency        leads to shorter wavelengths so that the sensing system can        detect scatterers of smaller diameters. For the illustrative        embodiment herein, this frequency is 360 KHz, which corresponds        to a wavelength of 0.942 mm.    -   3) Bandwidth: Bandwidth defines the frequency band around the        carrier frequency and it contributes directly to the        characteristic of the generated noise. Higher bandwidth makes        the transmitted noise signal to be highly variant in amplitude        through time. This makes the measurements more accurate as it        allows the correct correlation values to be much higher than        others. Besides, it defines the resolution of the sensing as it        constitutes as a metric that discriminates between two        scatterers present in the scene. The RMSE in relative range and        velocity estimation results is provided for different bandwidths        and window times in FIGS. 8A-D and 9A-D. As shown in both FIGS.        8A-D and 9A-D, for finest and coarse hypothetical range        resolution, in general, higher the bandwidth better the sensing        accuracy becomes. However, for coarse hypothetical velocity, the        accuracy of sensing is closely related to window time, and has a        more complex behavior as further elaborated below.

Algorithmic Parameters

-   -   4) Window Time: This parameter defines the time window that the        transmitted signal is divided into to be used for computing the        correlation values with the received signal. Increasing the        window time decreases the number of windows (i.e. the        measurements—see FIG. 11) and increase the computational cost        (see simulation time per measurement in FIG. 10). However        increasing it also increases the resolution of the hypothetical        velocity as we have more samples to scale in time (as in FIG.        12, the resolutions (step sizes) coincide up to a window time as        there are not enough samples to have a finer velocity        resolution). Looking at the simulation results, as expected, we        see the window time has similar effect with bandwidth, and hence        for finest and coarse hypothetical range resolution, higher        window time leads to a better sensing accuracy. However,        simulations show that for a coarser hypothetical velocity grid,        increasing the window time improves the accuracy up to a point.        If increased beyond this point, accuracy degrades. Nonetheless,        increasing the window time also increases the computational cost        and refresh rate for each measurement. Therefore, the window        time should be selected in view of the trade-offs.    -   5) Hypothetical Range and Velocity Grid: This grid defines the        resolutions of the hypothetical relative range and velocities to        compute the correlation values. As expected, the coarser the        resolution, the less the computational cost but also the worse        the sensing accuracy becomes. In order to overcome the        computational cost without sacrificing the sensing performance,        in an embodiment, a two-stage system (or multi-stage system)        with a cascaded coarse to fine search grid may be used. However,        for this to work efficiently, a region must be found which both        coarse and fine values lead to acceptable sensing results. In an        embodiment, considering various types of hypothetical grids, a        combination of window time and bandwidth is achieved with the        region shown with the dotted red rectangle on FIGS. 8A-D, 9A-D,        10 and 12. FIGS. 8A-D and 9A-D show that lowest root mean        squared error (RMSE) in relative range and velocity may be        achieved in this region for all types of hypothetical relative        range and velocity grid. FIG. 10 shows that since the window        time is relatively small (0.6-0.7 msec) in this embodiment, the        computational cost of producing the estimates is also in an        acceptable range and actually similar for the finest and        coarsest hypothetical search grids. FIG. 11 shows a number of        windows (measurements) for varying window times (sec) for        hypothetical relative range and velocity grid combinations. As        shown in FIG. 12, the finest hypothetical relative velocity step        size in the specified rectangle is very close to its minimum        possible value. Notice that as step size decreases, the        hypothetical relative velocity resolution increases.

Generally, in FIGS. 8A-D, 9A-D, 10 and 12, the red dotted rectangleshows the region with the best performance for all combinations. Whileinfinite variations are possible, in an embodiment, considering thewaveform and algorithmic parameters in terms of the computational costand sensing accuracy, the following configuration provided the requiredperformance:

-   -   Sampling Frequency: 1.0584 MHz    -   Carrier Frequency: 360 KHz    -   Wavelength: 0.942 mm    -   Bandwidth: 144 KHz-160 KHz    -   Window Time: 0.605-0.726 msec    -   Initial Measurement Delay: 1.194-1.315 msec    -   Hypothetical Relative Ran Grid: 0.32-1.6 mm (variable)        incremental steps from 1 cm to 10 cm.    -   Hypothetical Relative Velocity Grid: 0.219-0.5 m/sec (variable)        incremental steps from −20 m/sec to 20 m/sec.        The foregoing configuration is intended to be an example, not a        limitation. It will be apparent to a person of ordinary skill in        the art in view of the disclosure herein that the waveform and        algorithmic parameters should be evaluated for a given system,        both in terms of the computational cost and the desired sensing        accuracy, and that variation from the foregoing is expected from        system to system. Note that the illustrated parameters represent        one set of possible parameters which were chosen such that: 1)        they have similar parameters with the ultrasonic sensors on        market (bandwidth having some exceptions addressed below); 2)        they perform for the target detection, i.e. they will        simultaneously detect 2-D positions in sub-mm level and estimate        the relative velocities.        Sampling Frequency(f_(s)) and Bandwidth(BW)

As used herein, c0 refers to the speed of transmission in the medium,e.g., the speed of sound for ultrasonics, and the speed of light formm-wave. Dividing c0 by fs gives the target edge detection resolution.In CAF, the peaks showing the target location move their position withsteps of c0/fs. So increasing the sampling frequency decreases the stepsand increases the resolution in range. However, the width of the peaksin the CAF are important in determining the exact location of thetarget. The peaks move in more precise steps in accordance with fs, butthe width of the peaks are inversely proportional to bw. So to havesharp peaks at the correct location a larger BW is required.

In an embodiment, 1.0584 MHz>=fs>=340 KHz. Where fs=1.0584 MHz edgedetection resolution is about 0.00032 m or 0.32 mm. Higher samplingfrequencies may be used, however, for the same window time, using ahigher sampling frequency increases the computational complexity.Sub-millimeter edge detection can be achieved using fs=340 KHz. Thus, inan embodiment where sub-millimeter edge detection is required, fs may bebetween 340 KHz and 1.0584 MHz.

In an embodiment BW is at least 20 KHz. In an embodiment, BW=160 KHz. Inan embodiment, BW is at least 160 KHz. At 160 KHz, CAF peakresolution=c0/(2*BW)=1.06 mm, the selected value is the approximatewidth of the peaks that permits positioning position the target. Themaximum point of the peak resides within the 1 mm, which allowsidentification of the location of the target in sub-mm resolution.Higher BW will achieve better location, however, in the case ofcommercially available ultrasonic transducers, they typically achieve isaround 20 KHz. As higher bandwidth transducers become available (or aremade for this application), BW should increase. In an embodiment, usinga commercially available common inexpensive transducer, BW is lower than20 KHz. In the cases where BW is lower than the desired 160 KHz, signalprocessing approaches are pursued to enhance the peaks in the CAF toincrease the target detection resolution to the intended range.

Carrier frequency (fc) determines the wavelength through c0/fc˜=1 mmwhen fc=360 KHz, provided that the target being detected is not shorterthan the wavelength—otherwise the waves will ‘pass through’ and notreflect well. In a finger detection embodiment, 360 KhZ is sufficient.Using an fc lower than around 40 KHz is not recommended because, e.g., afinger can be a little less than 1 cm in width. In an embodiment, roomis provided for sufficient bandwidth around fc. In an illustrativeembodiment, bw=160 KHz and fs=340 KHz, and 120 KHz>=fc>=50 KHz. A personof ordinary skill in the art will see the relationship and the numerousvariations that are within the scope of the invention herein.

Window time and initial measurement delay are algorithmic parametersthat can also be controlled. In an embodiment, the window time andinitial measurement delay should not change as a result of shifting fc,fs and bw. Nonetheless, the range grid depends on the sampling samplingrate, whereas the velocity grid depends on the sampling rate and thewindow time. In an embodiment, the minimum for range grid starts from0.32 mm, which as will be apparent to a person of skill in the art, isdirectly related to the sub-mm edge detection level. The relativevelocity parameter depends on how precise the window can be resized(shrunk or stretched) to find a correct maximum in the CAF—a finervelocity resolution is available with a higher window time and samplingrate.

The illustrative parameters shown above appear to show a sweet spot withcurrently available ultrasonic transducers when all of theconsiderations herein are put together. Using available parts, thesensing capabilities can change in an undesirable way if the bandwidthis lowered, or use fc or fs lower than proposed. In an embodiment, theillustrative parameters may represent a best practical scenario for thedescribed target sensing purposes, given the parts that provide theseparameters.

In an embodiment, additional signal processing approaches can be pursuedif parts have are “worse” fc, BW, etc. As an example, a system usingtransducers with fc=40 KHz, BW=2 KHz, with sampling frequency set in 200KHz results as follows: wavelength=c0/fc=8.5 mm; edge detectionresolution=c0/fs=1.7 mm; and CAF peak resolution=c0/(2*BW)=8.5 cm. Insuch an embodiment, the located target is within an 8.5 cm width peak.In an embodiment, interpolation and enveloping the CAF provided a CAFmaximum which achieved 1-2 cm resolution.

Although examples herein are illustrated with reference to millimeterwave or ultrasonic embodiments, either may be used for the embodimentsdisclosed, illustrated and suggested herein.

Smartwatch Application

The sensing array, referred to herein as Reflectar sensor array,comprises either one Ultrasonic/mmWave receiver and twoultrasonic/mmWave transmitters, or one Ultrasonic/mmWave transmitter andtwo ultrasonic/mmWave receivers. In an embodiment, a transceiver (whichincludes a co-located ultrasonic/mmWave receiver and ultrasonic/mmWavetransmitter) may be used in lieu of one receiver and one transmitter. Inan embodiment, a Reflectar sensor array comprises a transceiver (i.e.,transmitter and co-located receiver), and a receiver located at adistance from the transceiver. In an embodiment, a Reflectar sensorarray comprises a transceiver and and a transmitter located at adistance from the transceiver. In an embodiment, a Reflectar sensorarray comprises at least two transceivers (rather than one transceiverand one receiver) located at a distance from each other. In numerousembodiments, additional receivers, transmitters and/or transceivers maybe employed. In an embodiment, additional receivers, transmitters and/ortransceivers are co-linear, or on the same plane, as the otherreceivers, transmitters and/or transceivers of the Reflectar sensorarray. In an embodiment, additional receivers, transmitters and/ortransceivers are non-co-linear, or not on the same plane, as the otherreceivers, transmitters and/or transceivers of the Reflectar sensorarray. The use of an additional non-co-linear receiver, transmitterand/or transceiver may increase resolution in the Z-axis. In anembodiment, the use of an additional non-co-linear receiver, transmitterand/or transceiver can be treated as having each combination having atleast one transmitter, at least one receiver and at least one moretransmitter or receiver (regardless of whether it is part of atransceiver), as its own Reflectar sensor array.

In an embodiment, a Reflectar array is placed on a side of a smartwatchor other wrist-wearable apparatus such that position and velocity of oneor more fingers can be tracked along the X- and Y-axes (and potentiallythe Z-axis as well), within a short range (e.g., 10 cm) extendingparallel to and outwards from the Reflectar sensing array. In anembodiment, Reflectar array can detect differential motion, such as theDoppler effect caused by rotation, which can be used to help detect anddistinguish gestures.

In an embodiment, when worn on a wrist-worn device with the Reflectarsensor's sensitivity directed toward the hand, the Reflectar sensorarray can be used to detect gestures made on the back of one's hand, asillustrated in FIGS. 13 and 14 a-b. In an embodiment, a Reflectar array(e.g., on a smartwatch) is used to recognize single finger gestures inthe x- and y-plane, such as:

-   -   1-finger left-to-right swipe gesture;    -   1-finger right-to-left swipe gesture;    -   1-finger top-to-bottom swipe gesture;    -   1-finger bottom-to-top swipe gesture;    -   1-finger single tap gesture;    -   1-finger double tap gesture;    -   1-finger long single tap gesture; and    -   1-finger drawing or inking.

In an embodiment, a Reflectar array (e.g., on a smartwatch) is used torecognize single finger gestures in the x- and y-plane, with some z-axissensing. In an embodiment, a Reflectar array (e.g., on a smartwatch) isused to recognize single finger gestures in the x- and y-plane, withsome z-axis sensing, such as:

-   -   1-finger left-to-right swipe gesture;    -   1-finger right-to-left swipe gesture;    -   1-finger top-to-bottom swipe gesture;    -   1-finger bottom-to-top swipe gesture;    -   1-finger single tap gesture;    -   1-finger double tap gesture;    -   1-finger long single tap gesture; and    -   1-finger drawing or inking.

In an embodiment, a Reflectar sensing array is used to recognizemulti-finger gestures including but not limited to:

-   -   ≥2-fingers making a left-to-right swipe gesture;    -   ≥2-fingers making a right-to-left swipe gesture;    -   ≥2-fingers making a top-to-bottom swipe gesture;    -   >2-fingers making a bottom-to-top swipe gesture;    -   >2-fingers making a single tap gesture;    -   >2-fingers making a double tap gesture;    -   >2-fingers making a long single tap gesture;    -   >2-fingers moving closer together;    -   >2-fingers moving further apart; and    -   >2-fingers drawing or inking.

In an embodiment, the Reflectar sensor array is used to detect motionsthat can be identified as part of a gesture, e.g., computationally. Inan embodiment, the Reflectar sensor array is used to detect touch usingits sensitivity to the Z-axis. Turning to FIG. 15, in an embodiment, inaddition to touch, the Reflectar sensor can detect gestural activitywithin the sensor range before touch, between touches and after touch.In an embodiment, the Reflectar sensor data can be used to classifygestural activity. In an embodiment, the Reflectar sensor data can beused to classify gestural activity as belonging to one or morecategories. In an embodiment, the Reflectar sensor data can be used toclassify gestural activity as belonging to one or more of the followingcategories: corner, circle, pigtail, zig-zag and spike. Other categoriesand classifications of gestural activity will be apparent to one ofskill in the art in view of this disclosure.

In an embodiment, a smartwatch or other computational device receivingdata from the Reflectar sensor array can recognize hybrid touch andin-air gestures along the X-, Y-, and Z-axes. In an embodiment, gesturescan involve single fingers, or extended objects such as multiplefingers, entire hands, and other appendages. In an embodiment, thedifferential relative velocity of the object, such as rotation, can beused to better identify and sense the gesturing object and theparticular gesture. In an embodiment moments of touch/contact are usedas a punctuation mark in gestural vocabularies. Below is presented asample gestural vocabulary and sample definitions. The samples areintended to provide examples to persons of skill in the art, but are notintended limit the scope of this disclosure or the proposed utilizationof the Reflectar sensor array:

-   -   In-air “corner” gesture between two touches        -   In an embodiment, these hybrid touch and in-air input            gestures could be used to select an area between two touch            points.        -   In an embodiment, these hybrid touch and in-air input            gestures could be used to select a text string between two            touch points.        -   In an embodiment, these hybrid touch and in-air input            gestures could be used to multi-select two discrete objects,            words, or controls between two touch points.    -   In-air “corner” gesture after contact        -   In an embodiment, these hybrid touch and in-air input            gestures can be used to switch the function or state of a            touched control after contact.    -   In-air “circle” gesture before contact        -   In an embodiment, these hybrid touch and in-air input            gestures could be used to change a control's function            quickly between two states by circling clockwise or            counterclockwise in-air prior to touching the control.    -   In-air “circle” gesture between contacts        -   In an embodiment, these hybrid touch and in-air input            gestures could be used to select an area between two touch            points.        -   In an embodiment, these hybrid touch and in-air input            gestures could be used to select a text string between two            touch points.        -   In an embodiment, these hybrid touch and in-air input            gestures could be used to multi-select two discrete objects,            words, or controls between two touch points.    -   In-air “circle” gesture after contact        -   In an embodiment, these hybrid touch and in-air input            gestures could be used to move linearly between the            selections of a graduated control including but not limited            to multi-state buttons, drop downs, dials, or sliders.    -   In-air “pigtail” gesture between contacts        -   In an embodiment, these hybrid touch and in-air input            gestures could be used to select an area between two touch            points.        -   In an embodiment, these hybrid touch and in-air input            gestures could be used to select a text string between two            touch points.        -   In an embodiment, these hybrid touch and in-air input            gestures could be used to multi-select two discrete objects,            words, or controls between two touch points.    -   In-air “pigtail” gesture after contact        -   In an embodiment, these hybrid touch and in-air input            gestures can be used to switch the function or state of a            touched control after contact.    -   In-air “zigzag” gesture between contacts        -   In an embodiment, these hybrid touch and in-air input            gestures could be used to select an area between two touch            points.        -   In an embodiment, these hybrid touch and in-air input            gestures could be used to select a text string between two            touch points.        -   In an embodiment, these hybrid touch and in-air input            gestures could be used to multi-select two discrete objects,            words, or controls between two touch points.    -   In-air “zigzag” gesture after contact        -   In an embodiment, these hybrid touch and in-air input            gestures could be used to delete or undo a sequential set of            control activations that were just performed prior to            sensing an in-air zigzag gesture.        -   In an embodiment, these hybrid touch and in-air input            gestures could be used to delete or undo a previously            selected area of on-screen content or controls.    -   In-air “spike” gesture before contact        -   In an embodiment, these hybrid touch and in-air input            gestures could be used as a proxy for the force of a control            activation or as a means to quickly toggle between two            possible input commands for a given control by gauging if            the in-air z-height of a finger's trajectory prior to a            control activation was above a certain threshold height or            slope.    -   In-air “spike” gesture between contacts        -   In an embodiment, these hybrid touch and in-air input            gestures could be used to link two or more discrete controls            or text strings together by in-air gestures of a certain            slope or height-threshold into a single activation or            selection.    -   In-air “spike” gesture after contact        -   In an embodiment, these hybrid touch and in-air input            gestures could be used to remove a previously inputted text            string.        -   In an embodiment, these hybrid touch and in-air input            gestures could be used to remove an on-screen control            including but not limited to a button, slider, dial, etc.        -   In an embodiment, these hybrid touch and in-air input            gestures could be used to remove a potential input command            among a set of input commands mapped to a given multi-state            control by first making contact with a control command            presently assigned to a given multi-state control and then            performing an in-air spike gesture to remove that option            from the multi-state control.

Additional Application

A Reflectar sensing array may be deployed on stationary or mobiledevices, such as, for example, laptop computers, tablets, phones,phablets, touch sensors (e.g., touch pads), a computer mouse, cameras,or even a desk or other surface (collectively, Reflectar-enableddevices).

In an embodiment, a Reflectar-enabled device incorporates a Reflectarsensing array consisting of ultrasonic/mmWave transmitters, receiverand/or transceivers integrated within or placed thereon. In anembodiment, a Reflectar sensing array may be placed on the corners oraround the periphery of a Reflectar-enabled device so that position andvelocity of one or more fingers can be tracked along the X-, Y-, andpossibly the Z-axes. In an embodiment, the ultrasonic/mmWavetransmitters, receiver and/or transceivers can be placed along the edge,similar to the wearable smartwatch application described above, so thatgestures along the side of the Reflectar-enabled device can be detectedand recognized. The geometry and sensing issues will be similar.

In an embodiment, when deployed on a Reflectar-enabled device, care mustbe taken to direct the Reflectar sensor's sensitivity toward therelevant area, i.e., the detected area. In an embodiment, one or more Inan embodiment, multiple Reflectar sensor arrays may be deployed on aReflectar-enabled device, permitting multiple, and potentiallyoverlapping areas of sensitivity/detection. Reflectar sensor arrays aredeployed on a Reflectar-enabled tablet to provide a virtual work surfaceas illustrated in FIG. 16. In an embodiment, multiple Reflectar sensorarrays are deployed to extend the virtual work surface shown in FIG. 16.In an embodiment, additional Reflectar sensor arrays are deployed toprovide additional areas of gesture detection, such as beside the tablet(not shown), to the tablet shown in FIG. 16. In an embodiment, one ormore Reflectar sensor arrays can be used to recognize single fingergestures in the x- and y-plane, with some z-axis sensing, including butnot limited to:

-   -   1-finger left-to-right swipe gesture;    -   1-finger right-to-left swipe gesture;    -   1-finger top-to-bottom swipe gesture;    -   1-finger bottom-to-top swipe gesture;    -   1-finger single tap gesture;    -   1-finger double tap gesture;    -   1-finger long single tap gesture; and    -   1-finger drawing or inking.

In an embodiment, one or more Reflectar sensor arrays can be used torecognize multi-finger gestures in a detection/sensitivity area of aReflectar-enabled device, including but not limited to:

-   -   ≥2-fingers making a left-to-right swipe gesture;    -   ≥2-fingers making a right-to-left swipe gesture;    -   ≥2-fingers making a top-to-bottom swipe gesture;    -   >2-fingers making a bottom-to-top swipe gesture;    -   >2-fingers making a single tap gesture;    -   >2-fingers making a double tap gesture;    -   >2-fingers making a long single tap gesture;    -   >2-fingers moving closer together;    -   >2-fingers moving further apart; and    -   >2-fingers drawing or inking.

In an embodiment, a Reflectar-enabled device (or other computationaldevice receiving data from the Reflectar sensor array) can recognizehybrid touch and in-air gestures along the X-, Y-, and Z-axes. In anembodiment, as described above, gestures can involve single fingers, orextended objects such as multiple fingers, entire hands, and otherappendages. In an embodiment, as described above, the differentialrelative velocity of the object, such as rotation, can be used to betteridentify and sense the gesturing object and the particular gesture. Inan embodiment moments of touch/contact are used as a punctuation mark ingestural vocabularies. Gestures may be classified within a gesturalvocabulary and interpreted according to definitions as described above.

Specialized vocabularies can be prepared for various states of, andvarious types of, Reflectar-enabled device. For example, for aReflectar-enabled phone, a gestural vocabulary around calls or textswould make sense. Similarly, a Reflectar-enabled phone, may reactdifferently to gestures when it is hand-held versus when it is on atable (or e.g., in a pocket). Similarly, a Reflectar-enabled tablet mayhave a different gestural vocabulary

Head-Mounted Display (“HMD”) Applications

In an embodiment, a head-mounted display (possibly designed for VR/ARapplications) is equipped (as discussed above) with ultrasonic ormm-wave transmitters and receivers so that the relative position andvelocity of nearby objects can be measured. In an embodiment, to enhancethe resolution of the measurements, the geometry of the TX/RX units canbe optimized as discussed above. In an embodiment, for a VR/ARapplication, the ultrasonic or mm-wave transmitters and receivers areplaced around the periphery of the headset such that object in front ofthe wearer can be sufficiently measured. In an embodiment, the relativeposition and movement of a hand in front of the headset would be thetarget of measurement, permitting certain hand gestures to berecognized. In an embodiment, the relative position and movement of auser's fingers and had in front of the headset would be the target ofmeasurement (i.e., having more detailed resolution), permitting furthergestures to be recognized. In an embodiment, the Doppler shift ismeasured and used in calculations (as discussed above) so that gesturesinvolving substantial movement, especially differential velocities(including but not limited to rotation gestures, differential swipinggestures, etc.) are easily distinguished from non-moving object. In anembodiment, the Doppler shift measurements are used to mitigate theeffects of radar/sonar clutter.

Hand-Held Controller Based Applications

In an embodiment, a handheld controller, such as a game controller, isequipped (as discussed above) with ultrasonic or mm-wave transmittersand receivers so that the relative position and velocity of nearbyobjects can be measured in a similar fashion as above. Because such acontroller is handheld, the finger position normal to the controllersurface of the holding hand will be measurable. In an embodiment,sensors can be placed under the fingers at the controller's grasppoints, allowing a height-above-controller-surface measurement. Theheight-above-controller-surface measurement allows, e.g., “trigger” andother finger gestures to be detected.

Combining HMD and Handheld Applications

In an embodiment, measurements from ultrasonic or mm-wave transmittersand receivers units mounted on an HMD measurements from a handheldcontroller are combined. Such combined measurement may yieldmeasurements better than can be achieved from either separately. Thecombination provides an extended sensor geometry, because of the extrasensors and their unique positions. In an embodiment, the relativepositions of the HMD and handheld controller should be known. In anembodiment, the relative positions of the HMD and handheld controllercan be measured directly e.g., with ultrasound or mm-wave, with a videosystem or some other RF, optical or acoustic measuring system. In anembodiment, the relative positions of the HMD and handheld controllercan be inferred from other sensors, such as inertial and magneticsensing devices.

Touch and Gesture Surface

With reference to FIG. 17, a touch and gestures surface having aplurality of TX and RX units around its periphery is illustrated. Atouch gesture surface can be created by placing TX and RX units aroundthe periphery of a two-dimensional waveguide. Note the waveguide can bea two-dimensional manifold of almost arbitrary shape. The TX units sendsignals into the waveguide and these signals are received and measuredby the RX units. In an embodiment, the signals are orthogonal to eachother in time, frequency, spread spectrum code, or some combination ofthese. The signals may be acoustic or electromagnetic in nature.

A touch on the surface of the manifold changes the propagationcharacteristics of the waveguide so that the received waveforms arechanged in some characteristic. In an embodiment, the propagation timeof the signals between a TX and an RX unit may change in some manner dueto a touch at a particular position on the surface. In an embodiment,the phase of the signals between a TX and an RX unit may change due to atouch at a particular position on the surface. In an embodiment, theamplitude of the signals between a TX and an RX unit may change due to atouch at a particular position on the surface. In an embodiment, thefrequency of the signals between a TX and an RX unit may change due to atouch or motion at a particular position on the surface. In anembodiment, the direction of propagation of the signals between a TX andan RX unit may change due to a touch or gesture at a particular locationon the surface.

The changes in these signals can be analyzed and, due to thecharacteristics of the particular waveguide, signals and TX and RXunits, along with how touches and gestures will affect the signals, theposition of the touch and/or nature of the gesture can be inferred fromthe signals.

In an embodiment, waveguide is selected based upon how it changes thesignals passing through it. In an embodiment, the selected waveguide thematerial enhance changes to the signals passing through it. In anembodiment, the selected waveguide the material optimizes the changes tothe signals passing through it. In an embodiment, a waveguide isselected to optimize the detection and recognition of desired touchesand/or gestures. In an embodiment, a waveguide is selected to optimizethe detection and recognition of only specific touches and/or gestures.In an embodiment, a waveguide is selected to mitigate the detection andrecognition of specific touches and/or gestures. In an embodiment, awaveguide is selected to mitigate propagation of at least some of thesignals in the absence of a touch. In an embodiment, the set of signalsis selected to optimize the detection and recognition of desired touchesand/or gestures. In an embodiment, the set of signals is selected tooptimize the detection and recognition of only specific touches and/orgestures. In an embodiment, the set of signals is selected to mitigatethe detection and recognition of specific touches and/or gestures. In anembodiment, the set of signals may be chosen to enhance and optimize thechanges to the signals passing through the waveguide, to optimize thedetection and recognition of touches and gestures.

In an embodiment, a waveguide is fitted with a plurality oftransmitters, receivers and/or transceivers along a periphery edge. Inan embodiment, the spacing between the components is 5 mm. In anembodiment, the spacing between the components is at least 7 mm. In anembodiment, the spacing between the components is at least 10 mm. In anembodiment, the spacing between the components is no more than 2.5 cm.In an embodiment, at least two transmitters and one receiver are placedon a periphery edge, and at least one transmitter and at least tworeceivers are placed on another periphery edge. It will be apparent toone of skill in the art, in view of this disclosure, that numerouscombinations of receivers and transmitters in various configurations maybe used, with the limitation that the transmitters provide signals thatcan propagate to at least one receiver, but in an embodiment to allreceivers in the absence of interaction with the waveguide. It will besimilarly apparent to one of skill in the art, in view of thisdisclosure, that numerous combinations of receivers and transmitters invarious configurations may be used, with the limitation that thereceivers can acquire signals from at least one transmitter, but in anembodiment, from all transmitters in the absence of interaction with thewaveguide.

The above embodiments and preferences are illustrative of the presentinvention. It is neither necessary, nor intended for this patent tooutline or define every possible combination or embodiment. The inventorhas disclosed sufficient information to permit one skilled in the art topractice at least one embodiment of the invention. The above descriptionand drawings are merely illustrative of the present invention and thatchanges in components, structure and procedure are possible withoutdeparting from the scope of the present invention as defined in thefollowing claims. For example, elements and/or steps described aboveand/or in the following claims in a particular order may be practiced ina different order without departing from the invention. Thus, while theinvention has been particularly shown and described with reference toembodiments thereof, it will be understood by those skilled in the artthat various changes in form and details may be made therein withoutdeparting from the spirit and scope of the invention.

We claim:
 1. A device for sensing movement of a close-range scatterercomprising: clock providing a clock signal; transmitter configured togenerate a first signal at a known time relative to the clock signal;first and second receivers spaced from one-another along a plane, thefirst and second receivers being adapted to receive echoed portions ofthe first signal during a receive window occurring during a known periodrelative to the clock signal; signal processor operatively connected tothe clock and the echoed portions of the first signal received by thefirst and second receivers, the signal processor configured to:determine, using a CAF, a first and second time-of-flight reflecting thetime between the known time of generating the first signal, and the timean echoed portion of the first signal is received by the first andsecond receivers, and calculating an X- and Y-coordinate on the planereflecting the X- and Y-location of a scatterer; determine a first timescaling reflecting the echoed portions of the first signal received bythe first receiver during the receive window, and calculating a movementbased on the determined first time scaling; determine a second timescaling reflecting the echoed portions of the first signal received bythe second receiver during the receive window, and calculating amovement based on the determined second time scaling; and outputting anX- and Y-coordinate and movement representing a scatterer from which thefirst signal was echoed.
 2. The device of claim 1, wherein the signalprocessor is configured to determine a first time scaling, and secondtime scaling using a CAF.
 3. The device of claim 1, wherein the deviceis a wearable device.
 4. The device of claim 3, wherein the device is awrist-wearable device.
 5. The device of claim 1, wherein the firstsignal is selected from the group of: an ultrasonic signal or a mm-wavesignal.
 6. The device of claim 1, wherein the first receiver and thetransmitter form a transducer.